Use of Digital COVID-19 Exposure Notifications at a Large Gathering: Survey Analysis of Public Health Conference Attendees.

BACKGROUND: WA Notify was Washington State's smartphone-based COVID-19 digital exposure notification (EN) tool, which was used to help limit the spread of COVID-19 between November 30, 2020, and May 11, 2023. Following the 2022 Washington State Public Health Association Annual Conference, attendees who had WA Notify activated began receiving ENs alerting them to a possible COVID-19 exposure during the conference. A survey was emailed to all conference attendees to measure WA Notify adoption, mechanisms through which attendees received ENs, and self-reported engagement in protective behaviors postexposure. OBJECTIVE: This study aimed to learn more about the experiences of WA Notify adopters and nonadopters who may have been exposed to COVID-19 at a large group gathering. METHODS: A web-based survey administered through REDCap (Research Electronic Data Capture; Vanderbilt University) was sent to all attendees of the Washington State Public Health Association conference. Self-reported demographic information and characteristics of respondents were summarized. Regression models were used to estimate relative risks to compare WA Notify adoption and testing behaviors between groups. RESULTS: Of the 464 total registered attendees who were sent the survey, 205 (44%) responses were received; 201 eligible attendees were included in this analysis. Of those, 149 (74%) respondents reported having WA Notify activated on their phones at the time of the conference. Among respondents with WA Notify activated, 54% (n=77) reported learning of their potential exposure from a WA Notify EN. Respondents who reported that they did not have WA Notify activated and learned of their potential exposure via the event-wide email from conference organizers were 39% less likely to test for COVID-19 compared to respondents with WA Notify activated who learned of their potential exposure from the email (relative risk 0.61, 95% CI 0.40-0.93; P=.02), and this gap was even larger when compared to respondents who learned of their exposure from a WA Notify EN. The most commonly cited reason for not having WA Notify activated was privacy concerns (n=17, 35%), followed by not wanting to receive ENs (n=6, 12%) and being unaware of WA Notify (n=5, 10%). CONCLUSIONS: Digital EN systems are an important tool to directly and anonymously notify close contacts of potential exposures and provide guidance on the next steps in a timely manner. Given the privacy concerns, there is still a need for increasing transparency surrounding EN technology to increase uptake by the public if this technology were to be used in the future to slow the spread of communicable diseases.

Venous Thromboembolism Among People With HIV: Design, Implementation, and Findings of a Centralized Adjudication System in Clinical Care Sites Across the United States.

BACKGROUND: People with HIV (PWH) are at increased risk for venous thromboembolism (VTE). We conducted this study to characterize VTE including provoking factors among PWH in the current treatment era. METHODS: We included PWH with VTE between 2010 and 2020 at 6 sites in the CFAR Network of Integrated Clinical Systems cohort. We ascertained for possible VTE using diagnosis, VTE-related imaging, and VTE-related procedure codes, followed by centralized adjudication of primary data by expert physician reviewers. We evaluated sensitivity and positive predictive value of VTE ascertainment approaches. VTEs were classified by type and anatomic location. Reviewers identified provoking factors such as hospitalizations, infections, and other potential predisposing factors such as smoking. RESULTS: We identified 557 PWH with adjudicated VTE: 239 (43%) had pulmonary embolism with or without deep venous thrombosis, and 318 (57%) had deep venous thrombosis alone. Ascertainment with clinical diagnoses alone missed 6% of VTEs identified with multiple ascertainment approaches. DVTs not associated with intravenous lines were most often in the proximal lower extremities. Among PWH with VTE, common provoking factors included recent hospitalization (n = 134, 42%), infection (n = 133, 42%), and immobilization/bed rest (n = 78, 25%). Only 57 (10%) PWH had no provoking factor identified. Smoking (46%), HIV viremia (27%), and injection drug use (22%) were also common. CONCLUSIONS: We conducted a robust adjudication process that demonstrated the benefits of multiple ascertainment approaches followed by adjudication. Provoked VTEs were more common than unprovoked events. Nontraditional and modifiable potential predisposing factors such as viremia and smoking were common.

Tobacco Smoking and Pack-Years Are Associated With Frailty Among People With HIV.

BACKGROUND: Tobacco smoking increases frailty risk among the general population and is common among people with HIV (PWH) who experience higher rates of frailty at younger ages than the general population. METHODS: We identified 8608 PWH across 6 Centers for AIDS Research Network of Integrated Clinical Systems sites who completed ≥2 patient-reported outcome assessments, including a frailty phenotype measuring unintentional weight loss, poor mobility, fatigue, and inactivity, and scored 0-4. Smoking was measured as baseline pack-years and time-updated never, former, or current use with cigarettes/day. We used Cox models to associate smoking with risk of incident frailty (score ≥3) and deterioration (frailty score increase by ≥2 points), adjusted for demographics, antiretroviral medication, and time-updated CD4 count. RESULTS: The mean follow-up of PWH was 5.3 years (median: 5.0), the mean age at baseline was 45 years, 15% were female, and 52% were non-White. At baseline, 60% reported current or former smoking. Current (HR: 1.79; 95% confidence interval: 1.54 to 2.08) and former (HR: 1.31; 95% confidence interval: 1.12 to 1.53) smoking were associated with higher incident frailty risk, as were higher pack-years. Current smoking (among younger PWH) and pack-years, but not former smoking, were associated with higher risk of deterioration. CONCLUSIONS: Among PWH, smoking status and duration are associated with incident and worsening frailty.

Preliminary Efficacy of Let's Talk Tech: Technology Use Planning for Dementia Care Dyads.

Background and Objective: Care partners of people living with dementia require support to knowledgeably navigate decision making about how and when to use monitoring technologies for care purposes. We conducted a pilot study of a novel self-administered intervention, "Let's Talk Tech," for people living with mild dementia and their care partners. This paper presents preliminary efficacy findings of this intervention designed to educate and facilitate dyadic communication about a range of technologies used in dementia care and to document the preferences of the person living with dementia. It is the first-of-its-kind decision-making and planning tool with a specific focus on technology use. Research Design and Methods: We used a 1-group pretest-post-test design and paired t tests to assess change over 2 time periods in measures of technology comprehension, care partner knowledge of the participant living with mild Alzheimer's disease's (AD) preferences, care partner preparedness to make decisions about technology use, and mutual understanding. Thematic analysis was conducted on postintervention interview transcripts to elucidate mechanisms and experiences with Let's Talk Tech. Results: Twenty-nine mild AD dementia care dyads who live together completed the study. There was statistically significant improvement with medium and large effect sizes on outcome measures of care partners' understanding of each technology, care partners' perceptions of the person living with dementia's understanding of each technology, knowledge of the person living with dementia's preferences, decision-making preparedness, and care partners' feelings of mutual understanding. Participants reported that it helped them have important and meaningful conversations about using technology. Discussion and Implications: Let's Talk Tech demonstrated promising preliminary efficacy on targeted measures that can lead to informed, shared decision making about technologies used in dementia care. Future studies should assess efficacy with larger samples and more diverse sample populations in terms of race, ethnicity, and dementia type.

Intervention to Promote Communication About Goals of Care for Hospitalized Patients With Serious Illness: A Randomized Clinical Trial.

Importance: Discussions about goals of care are important for high-quality palliative care yet are often lacking for hospitalized older patients with serious illness. Objective: To evaluate a communication-priming intervention to promote goals-of-care discussions between clinicians and hospitalized older patients with serious illness. Design, Setting, and Participants: A pragmatic, randomized clinical trial of a clinician-facing communication-priming intervention vs usual care was conducted at 3 US hospitals within 1 health care system, including a university, county, and community hospital. Eligible hospitalized patients were aged 55 years or older with any of the chronic illnesses used by the Dartmouth Atlas project to study end-of-life care or were aged 80 years or older. Patients with documented goals-of-care discussions or a palliative care consultation between hospital admission and eligibility screening were excluded. Randomization occurred between April 2020 and March 2021 and was stratified by study site and history of dementia. Intervention: Physicians and advance practice clinicians who were treating the patients randomized to the intervention received a 1-page, patient-specific intervention (Jumpstart Guide) to prompt and guide goals-of-care discussions. Main Outcomes and Measures: The primary outcome was the proportion of patients with electronic health record-documented goals-of-care discussions within 30 days. There was also an evaluation of whether the effect of the intervention varied by age, sex, history of dementia, minoritized race or ethnicity, or study site. Results: Of 3918 patients screened, 2512 were enrolled (mean age, 71.7 [SD, 10.8] years and 42% were women) and randomized (1255 to the intervention group and 1257 to the usual care group). The patients were American Indian or Alaska Native (1.8%), Asian (12%), Black (13%), Hispanic (6%), Native Hawaiian or Pacific Islander (0.5%), non-Hispanic (93%), and White (70%). The proportion of patients with electronic health record-documented goals-of-care discussions within 30 days was 34.5% (433 of 1255 patients) in the intervention group vs 30.4% (382 of 1257 patients) in the usual care group (hospital- and dementia-adjusted difference, 4.1% [95% CI, 0.4% to 7.8%]). The analyses of the treatment effect modifiers suggested that the intervention had a larger effect size among patients with minoritized race or ethnicity. Among 803 patients with minoritized race or ethnicity, the hospital- and dementia-adjusted proportion with goals-of-care discussions was 10.2% (95% CI, 4.0% to 16.5%) higher in the intervention group than in the usual care group. Among 1641 non-Hispanic White patients, the adjusted proportion with goals-of-care discussions was 1.6% (95% CI, -3.0% to 6.2%) higher in the intervention group than in the usual care group. There was no evidence of differential treatment effects of the intervention on the primary outcome by age, sex, history of dementia, or study site. Conclusions and Relevance: Among hospitalized older adults with serious illness, a pragmatic clinician-facing communication-priming intervention significantly improved documentation of goals-of-care discussions in the electronic health record, with a greater effect size in racially or ethnically minoritized patients. Trial Registration: ClinicalTrials.gov Identifier: NCT04281784.

Time Utilization Among Immunization Clinics Using an Electronic Immunization Registry (Part 2): Time and Motion Study of Modified User Workflows.

BACKGROUND: Digital health interventions have the potential to improve the provision of health care services through digitized data collection and management. Low- and middle-income countries are beginning to introduce electronic immunization registries (EIRs) into their routine immunization services to better capture and store childhood vaccination information. Especially in Africa, where 25% of children remain unimmunized or underimmunized, technologies that can help identify children due for a vaccination are particularly important for improving vaccination coverage. However, an improved understanding of the effectiveness of these systems is needed to develop and deploy sustainable EIRs in low- and middle-income countries. OBJECTIVE: We conducted an interventional pretest-posttest design study that sought to improve time efficiency through workflow modifications in Kenyan immunization clinics. Our aim was to describe how activity times differed after introducing workflow modifications that could potentially reduce the time needed to perform routine data entry activities. Our intent was to demonstrate changes in efficiency when moving from the existing dual-data entry workflow to a future paperless workflow by health facility size and experience length of health care workers (HCWs). METHODS: We tested how 3 workflow modifications would affect time utilization among HCWs using the EIR at the point of care compared with baseline immunization clinic workflows. Our outcome of interest was the time taken to complete individual activities and a patient's total time in the clinic where we compared the time spent during the baseline workflow with that during the modified workflow. We used a standardized tool to observe and document the immunization clinic workflow. To estimate differences in time utilization, we used bivariate analyses and fit multivariate linear mixed-effects models. RESULTS: Our study found that for HCWs using an EIR, the introduction of modified workflows decreased the amount of time needed to provide services to children seen in the immunization clinic. With a baseline mean time of 10 minutes spent per child, this decreased by about 3 minutes when the preparation modification was introduced and almost 5 minutes for the paperless and combined modifications. Results pertaining to the EIR's performance and ability to connect to the internet were particularly insightful about potential causes of delays. CONCLUSIONS: We were able to conduct a concise clinical simulation exercise by introducing modified workflows and estimating their impact on time utilization in immunization clinics using an EIR. We found that the paperless workflow provided the largest time savings when delivering services, although this was threatened by poor EIR performance and internet connectivity. This study demonstrated that not only should digital health interventions be built and adapted for particular use cases but existing user workflows also need to adapt to new technology.

Integration of a Digital Health Intervention Into Immunization Clinic Workflows in Kenya: Qualitative, Realist Evaluation of Technology Usability.

BACKGROUND: In an effort to increase vaccination coverage in low-resource settings, digital tools have been introduced to better track immunization records, improve data management practices, and provide improved access to vaccination coverage data for decision-making. Despite the potential of these electronic systems to improve the provision of health services, few digital health interventions have been institutionalized at scale in low- and middle-income countries. OBJECTIVE: In this paper, we aimed to describe how health care workers in Kenya had integrated an electronic immunization registry into their immunization clinic workflows and to use these findings to inform the development of a refined program theory on the registry's usability. METHODS: Informed by realist methodology, we developed a program theory to explain usability of the electronic immunization registry. We designed a qualitative study based on our theory to describe the barriers and facilitators influencing data entry and use. Qualitative data were collected through semistructured interviews with users and workflow observations of immunization clinic sessions. Our findings were summarized by context-mechanism-outcome relationships formed after analyzing our key themes across interviews and workflow observations. Using these relationships, we were able to identify common rules for future implementers. RESULTS: Across the 12 facilities included in our study, 19 health care workers were interviewed, and 58 workflow sessions were observed. The common rules developed from our qualitative findings are as follows: rule 1-ensure that the users complete training to build familiarity with the system, understand the value of the system and data, and know where to find support; rule 2-confirm that the system captures all data needed for users to provide routine health care services and is easy to navigate; rule 3-identify work-arounds for poor network, system performance, and too few staff or resources; and rule 4-make users aware of expected changes to their workflow, and how these changes might differ over time and by facility size or number of patients. Upon study completion, we revised the program theory to reflect the importance of the goals and workflows of electronic immunization registries aligning with reality. CONCLUSIONS: We created a deeper understanding of the underlying mechanisms for usability of the registry. We found that the electronic immunization registry had high acceptability among users; however, there were numerous barriers to using the system, even under ideal conditions, causing a misalignment between the system and the reality of the users' workflows and their environment. Human-centered design and human-factors methods can assist during pilot stages to better align systems with users' needs and again after scale-up to ensure that interventions are suitable for all user settings.

Associations between alcohol and cigarette use and type 1 and 2 myocardial infarction among people with HIV.

OBJECTIVES: People with HIV have a higher risk of myocardial infarction (MI) than the general population, with a greater proportion of type 2 MI (T2MI) due to oxygen demand-supply mismatch compared with type 1 (T1MI) resulting from atherothrombotic plaque disruption. People living with HIV report a greater prevalence of cigarette and alcohol use than do the general population. Alcohol use and smoking as risk factors for MI by type are not well studied among people living with HIV. We examined longitudinal associations between smoking and alcohol use patterns and MI by type among people living with HIV. DESIGN AND METHODS: Using longitudinal data from the Centers for AIDS Research Network of Integrated Clinical Systems cohort, we conducted time-updated Cox proportional hazards models to determine the impact of smoking and alcohol consumption on adjudicated T1MI and T2MI. RESULTS: Among 13 506 people living with HIV, with a median 4 years of follow-up, we observed 177 T1MI and 141 T2MI. Current smoking was associated with a 60% increase in risk of both T1MI and T2MI. In addition, every cigarette smoked per day was associated with a 4% increase in risk of T1MI, with a suggestive, but not significant, 2% increase for T2MI. Cigarette use had a greater impact on T1MI for men than for women and on T2MI for women than for men. Increasing alcohol use was associated with a lower risk of T1MI but not T2MI. Frequency of heavy episodic alcohol use was not associated with MI. CONCLUSIONS: Our findings reinforce the prioritization of smoking reduction, even without cessation, and cessation among people living with HIV for MI prevention and highlight the different impacts on MI type by gender.

Assessment of Natural Language Processing of Electronic Health Records to Measure Goals-of-Care Discussions as a Clinical Trial Outcome.

Importance: Many clinical trial outcomes are documented in free-text electronic health records (EHRs), making manual data collection costly and infeasible at scale. Natural language processing (NLP) is a promising approach for measuring such outcomes efficiently, but ignoring NLP-related misclassification may lead to underpowered studies. Objective: To evaluate the performance, feasibility, and power implications of using NLP to measure the primary outcome of EHR-documented goals-of-care discussions in a pragmatic randomized clinical trial of a communication intervention. Design, Setting, and Participants: This diagnostic study compared the performance, feasibility, and power implications of measuring EHR-documented goals-of-care discussions using 3 approaches: (1) deep-learning NLP, (2) NLP-screened human abstraction (manual verification of NLP-positive records), and (3) conventional manual abstraction. The study included hospitalized patients aged 55 years or older with serious illness enrolled between April 23, 2020, and March 26, 2021, in a pragmatic randomized clinical trial of a communication intervention in a multihospital US academic health system. Main Outcomes and Measures: Main outcomes were natural language processing performance characteristics, human abstractor-hours, and misclassification-adjusted statistical power of methods of measuring clinician-documented goals-of-care discussions. Performance of NLP was evaluated with receiver operating characteristic (ROC) curves and precision-recall (PR) analyses and examined the effects of misclassification on power using mathematical substitution and Monte Carlo simulation. Results: A total of 2512 trial participants (mean [SD] age, 71.7 [10.8] years; 1456 [58%] female) amassed 44 324 clinical notes during 30-day follow-up. In a validation sample of 159 participants, deep-learning NLP trained on a separate training data set identified patients with documented goals-of-care discussions with moderate accuracy (maximal F1 score, 0.82; area under the ROC curve, 0.924; area under the PR curve, 0.879). Manual abstraction of the outcome from the trial data set would require an estimated 2000 abstractor-hours and would power the trial to detect a risk difference of 5.4% (assuming 33.5% control-arm prevalence, 80% power, and 2-sided α = .05). Measuring the outcome by NLP alone would power the trial to detect a risk difference of 7.6%. Measuring the outcome by NLP-screened human abstraction would require 34.3 abstractor-hours to achieve estimated sensitivity of 92.6% and would power the trial to detect a risk difference of 5.7%. Monte Carlo simulations corroborated misclassification-adjusted power calculations. Conclusions and Relevance: In this diagnostic study, deep-learning NLP and NLP-screened human abstraction had favorable characteristics for measuring an EHR outcome at scale. Adjusted power calculations accurately quantified power loss from NLP-related misclassification, suggesting that incorporation of this approach into the design of studies using NLP would be beneficial.

Development of Frail RISC-HIV: a Risk Score for Predicting Frailty Risk in the Short-term for Care of People with HIV.

OBJECTIVE: Frailty is common among people with HIV (PWH), so we developed frail risk in the short-term for care (RISC)-HIV, a frailty prediction risk score for HIV clinical decision-making. DESIGN: We followed PWH for up to 2 years to identify short-term predictors of becoming frail. METHODS: We predicted frailty risk among PWH at seven HIV clinics across the United States. A modified self-reported Fried Phenotype captured frailty, including fatigue, weight loss, inactivity, and poor mobility. PWH without frailty were separated into training and validation sets and followed until becoming frail or 2 years. Bayesian Model Averaging (BMA) and five-fold-cross-validation Lasso regression selected predictors of frailty. Predictors were selected by BMA if they had a greater than 45% probability of being in the best model and by Lasso if they minimized mean squared error. We included age, sex, and variables selected by both BMA and Lasso in Frail RISC-HIV by associating incident frailty with each selected variable in Cox models. Frail RISC-HIV performance was assessed in the validation set by Harrell's C and lift plots. RESULTS: Among 3170 PWH (training set), 7% developed frailty, whereas among 1510 PWH (validation set), 12% developed frailty. BMA and Lasso selected baseline frailty score, prescribed antidepressants, prescribed antiretroviral therapy, depressive symptomology, and current marijuana and illicit opioid use. Discrimination was acceptable in the validation set, with Harrell's C of 0.76 (95% confidence interval: 0.73-0.79) and sensitivity of 80% and specificity of 61% at a 5% frailty risk cutoff. CONCLUSIONS: Frail RISC-HIV is a simple, easily implemented tool to assist in classifying PWH at risk for frailty in clinics.

Validity Properties of a Self-reported Modified Frailty Phenotype Among People With HIV in Clinical Care in the United States.

ABSTRACT: Modifications to Fried's frailty phenotype (FFP) are common. We evaluated a self-reported modified frailty phenotype (Mod-FP) used among people with HIV (PWH). Among 522 PWH engaged in two longitudinal studies, we assessed validity of the four-item Mod-FP compared with the five-item FFP. We compared the phenotypes via receiver operator characteristic curves, agreement in classifying frailty, and criterion validity via association with having experienced falls. Mod-FP classified 8% of PWH as frail, whereas FFP classified 9%. The area under the receiver operator characteristic curve for Mod-FP classifying frailty was 0.93 (95% CI = 0.91-0.96). We observed kappa ranging from 0.64 (unweighted) to 0.75 (weighted) for categorizing frailty status. Both definitions found frailty associated with a greater odds of experiencing a fall; FFP estimated a slightly greater magnitude (i.e., OR) for the association than Mod-FP. The Mod-FP has good performance in measuring frailty among PWH and is reasonable to use when the gold standards of observed assessments (i.e., weakness and slowness) are not feasible.

Predictors of Documented Goals-of-Care Discussion for Hospitalized Patients With Chronic Illness.

CONTEXT: Goals-of-care discussions are important for patient-centered care among hospitalized patients with serious illness. However, there are little data on the occurrence, predictors, and timing of these discussions. OBJECTIVES: To examine the occurrence, predictors, and timing of electronic health record (EHR)-documented goals-of-care discussions for hospitalized patients. METHODS: This retrospective cohort study used natural language processing (NLP) to examine EHR-documented goals-of-care discussions for adults with chronic life-limiting illness or age ≥80 hospitalized 2015-2019. The primary outcome was NLP-identified documentation of a goals-of-care discussion during the index hospitalization. We used multivariable logistic regression to evaluate associations with baseline characteristics. RESULTS: Of 16,262 consecutive, eligible patients without missing data, 5,918 (36.4%) had a documented goals-of-care discussion during hospitalization; approximately 57% of these discussions occurred within 24 hours of admission. In multivariable analysis, documented goals-of-care discussions were more common for women (OR=1.26, 95%CI 1.18-1.36), older patients (OR=1.04 per year, 95%CI 1.03-1.04), and patients with more comorbidities (OR=1.11 per Deyo-Charlson point, 95%CI 1.10-1.13), cancer (OR=1.88, 95%CI 1.72-2.06), dementia (OR=2.60, 95%CI 2.29-2.94), higher acute illness severity (OR=1.12 per National Early Warning Score point, 95%CI 1.11-1.14), or prior advance care planning documents (OR=1.18, 95%CI 1.08-1.30). Documentation of these discussions was less common for racially or ethnically minoritized patients (OR=0.823, 95%CI 0.75-0.90). CONCLUSION: Among hospitalized patients with serious illness, documented goals-of-care discussions identified by NLP were more common among patients with older age and increased burden of acute or chronic illness, and less common among racially or ethnically minoritized patients. This suggests important disparities in goals-of-care discussions.

Trading-off privacy and utility: the Washington State experience assessing the performance of a public health digital exposure notification system for coronavirus disease 2019.

OBJECTIVE: Digital exposure notifications (DEN) systems were an emergency response to the coronavirus disease 2019 (COVID-19) pandemic, harnessing smartphone-based technology to enhance conventional pandemic response strategies such as contact tracing. We identify and describe performance measurement constructs relevant to the implementation of DEN tools: (1) reach (number of users enrolled in the intervention); (2) engagement (utilization of the intervention); and (3) effectiveness in preventing transmissions of COVID-19 (impact of the intervention). We also describe WA State's experience utilizing these constructs to design data-driven evaluation approaches. METHODS: We conducted an environmental scan of DEN documentation and relevant publications. Participation in multidisciplinary collaborative environments facilitated shared learning. Compilation of available data sources and their relevance to implementation and operation workflows were synthesized to develop implementation evaluation constructs. RESULTS: We identified 8 useful performance indicators within reach, engagement, and effectiveness constructs. DISCUSSION: We use implementation science to frame the evaluation of DEN tools by linking the theoretical constructs with the metrics available in the underlying disparate, deidentified, and aggregate data infrastructure. Our challenges in developing meaningful metrics include limited data science competencies in public health, validation of analytic methodologies in the complex and evolving pandemic environment, and the lack of integration with the public health infrastructure. CONCLUSION: Continued collaboration and multidisciplinary consensus activities can improve the utility of DEN tools for future public health emergencies.

Changes in on-time vaccination following the introduction of an electronic immunization registry, Tanzania 2016-2018: interrupted time-series analysis.

BACKGROUND: Digital health interventions (DHI) have the potential to improve the management and utilization of health information to optimize health care worker performance and provision of care. Despite the proliferation of DHI projects in low-and middle-income countries, few have been evaluated in an effort to understand their impact on health systems and health-related outcomes. Although more evidence is needed on their impact and effectiveness, the use of DHIs among immunization programs has become more widespread and shows promise for improving vaccination uptake and adherence to immunization schedules. METHODS: Our aim was to assess the impact of an electronic immunization registry (EIR) using an interrupted time-series analysis to analyze the effect on proportion of on-time vaccinations following introduction of an EIR in Tanzania. We hypothesized that the introduction of the EIR would lead to statistically significant changes in vaccination timeliness at 3, 6, and > 6 months post-introduction. RESULTS: For our primary analysis, we observed a decrease in the proportion of on-time vaccinations following EIR introduction. In contrast, our sensitivity analysis estimated improvements in timeliness among those children with complete vaccination records. However, we must emphasize caution interpreting these findings as they are likely affected by implementation challenges. CONCLUSIONS: This study highlights the complexities of using digitized individual-level routine health information system data for evaluation and research purposes. EIRs have the potential to improve vaccination timeliness, but analyses using EIR data can be complicated by data quality issues and inconsistent data entry leading to difficulties interpreting findings.

Risk Factors for Suboptimal Adherence Identified by Patient-Reported Outcomes Assessments in Routine HIV Care at 2 North American Clinics.

Purpose: Use of patient-reported outcomes assessments (PROs) can improve patient-provider communication and focus provider attention on current health issues. This analysis examines the association between suboptimal antiretroviral therapy (ART) adherence and factors obtained through PROs among people with HIV (PWH) at 2 North American outpatient clinics. Patients and Methods: Immediately before a clinic visit, PWH completed self-administered PROs. Unadjusted and adjusted odds ratios (ORs) and 95% confidence intervals (CIs) were estimated from logistic regression models to identify sociodemographic and health-related factors (satisfaction with ART, difficulty meeting housing costs, depression, intimate partner violence, risk of malnutrition, smoking status, alcohol use, and substance use) associated with suboptimal adherence (defined as self-reporting <95% or <80% adherence). Multiple imputation was performed to account for missing data in the multivariate analyses. Results: Of 1632 PWH, 1239 (76%) responded to the adherence assessment; of these, 268 (22%) and 106 (9%) reported <95% and <80% adherence, respectively. Of 1580 PWH who responded, 354 (22%) were dissatisfied with their HIV medication. Of responding PWH, 19% reported moderate-to-severe depression, 23% indicated they were at risk of malnutrition, 34% were current smokers, and 62% reported substance use in the past 3 months. Dissatisfaction with ART was significantly associated with <95% and <80% adherence in the unadjusted analysis (unadjusted OR [95% CI], 3.38 [2.51-4.56] and 4.26 [2.82-6.42], respectively) and adjusted analysis (adjusted OR [95% CI], 2.76 [1.91-4.00] and 3.28 [1.95-5.52], respectively); significance remained after multiple imputation. In adjusted analyses, no risk of malnutrition was significantly associated with reduced odds of <95% adherence after multiple imputation (adjusted OR [95% CI], 0.714 [0.511-0.997]); no other factors were associated with <95% or <80% adherence. Conclusion: These results suggest that implementation of PROs evaluating treatment satisfaction may provide value to adherence management in routine HIV care.

Take on transplant: human-centered design of a patient education tool to facilitate informed discussions about lung transplant among people with cystic fibrosis.

OBJECTIVE: Lung transplant (LTx) saves lives in cystic fibrosis (CF). However, many potential candidates express uncertainty about LTx and die before receiving this treatment. CF guidelines recommend LTx education and clinical discussions well before the need for LTx arises, but limited patient resources exist. MATERIALS AND METHODS: We engaged people with CF and CF physicians in human-centered design of "Take On Transplant" (TOT), a web-based education tool to prepare patients for LTx discussions. Across 3 phases, needs assessment, design groups, and iterative user testing of TOT, we refined TOT from wireframe prototypes, to an interactive website, to a fully functional intervention ready for clinical trials. RESULTS: Fifty-five people with CF and 105 physicians identified information needs to prepare for LTx discussions. Design groups (n = 14 participants) then established core requirements: didactic education ("Resource Library"), patient narratives ("CF Stories"), frequently asked questions ("FAQ"), and self-assessment to tailor content ("My CF Stage"). Iterative usability testing (n = 39) optimized the design of CF Stories and prototype layout. We then developed the TOT website and demonstrated feasibility and preliminary efficacy of use through 2-week field testing (n = 9). DISCUSSION: Our human-centered design process provided guidance for educational tools to serve the evolving needs of potential LTx candidates. Our findings support the process of patient deliberation as a foundation for shared decision-making in CF, and inform educational tools that could potentially translate beyond LTx. CONCLUSION: TOT fills a critical gap in preparing people with CF for shared decision-making about LTx and may serve as a model for educational tools for other preference-sensitive decisions.

Tobacco smoking and binge alcohol use are associated with incident venous thromboembolism in an HIV cohort.

BACKGROUND: People with HIV (PWH) are at increased risk of cardiovascular comorbidities and substance use is a potential predisposing factor. We evaluated associations of tobacco smoking and alcohol use with venous thromboembolism (VTE) in PWH. METHODS: We assessed incident, centrally adjudicated VTE among 12 957 PWH within the Centers for AIDS Research Network of Integrated Clinical Systems (CNICS) cohort between January 2009 and December 2018. Using separate Cox proportional hazards models, we evaluated associations of time-updated alcohol and cigarette use with VTE, adjusting for demographic and clinical characteristics. Smoking was evaluated as pack-years and never, former, or current use with current cigarettes per day. Alcohol use was parameterized using categorical and continuous alcohol use score, frequency of use, and binge frequency. RESULTS: During a median of 3.6 years of follow-up, 213 PWH developed a VTE. One-third of PWH reported binge drinking and 40% reported currently smoking. In adjusted analyses, risk of VTE was increased among both current (HR: 1.44, 95% CI: 1.02-2.03) and former (HR: 1.44, 95% CI: 0.99-2.07) smokers compared to PWH who never smoked. Additionally, total pack-years among ever-smokers (HR: 1.10 per 5 pack-years; 95% CI: 1.03-1.18) was associated with incident VTE in a dose-dependent manner. Frequency of binge drinking was associated with incident VTE (HR: 1.30 per 7 days/month, 95% CI: 1.11-1.52); however, alcohol use frequency was not. Severity of alcohol use was not significantly associated with VTE. CONCLUSIONS: Current smoking and pack-year smoking history were dose-dependently associated with incident VTE among PWH in CNICS. Binge drinking was also associated with VTE. Interventions for smoking and binge drinking may decrease VTE risk among PWH.

Mixed-methods evaluation of three natural language processing modeling approaches for measuring documented goals-of-care discussions in the electronic health record.

CONTEXT: Documented goals-of-care discussions are an important quality metric for patients with serious illness. Natural language processing (NLP) is a promising approach for identifying goals-of-care discussions in the electronic health record (EHR). OBJECTIVES: To compare three NLP modeling approaches for identifying EHR documentation of goals-of-care discussions and generate hypotheses about differences in performance. METHODS: We conducted a mixed-methods study to evaluate performance and misclassification for three NLP featurization approaches modeled with regularized logistic regression: bag-of-words (BOW), rule-based, and a hybrid approach. From a prospective cohort of 150 patients hospitalized with serious illness over 2018 to 2020, we collected 4391 inpatient EHR notes; 99 (2.3%) contained documented goals-of-care discussions. We used leave-one-out cross-validation to estimate performance by comparing pooled NLP predictions to human abstractors with receiver-operating-characteristic (ROC) and precision-recall (PR) analyses. We qualitatively examined a purposive sample of 70 NLP-misclassified notes using content analysis to identify linguistic features that allowed us to generate hypotheses underpinning misclassification. RESULTS: All three modeling approaches discriminated between notes with and without goals-of-care discussions (AUCROC: BOW, 0.907; rule-based, 0.948; hybrid, 0.965). Precision and recall were only moderate (precision at 70% recall: BOW, 16.2%; rule-based, 50.4%; hybrid, 49.3%; AUCPR: BOW, 0.505; rule-based, 0.579; hybrid, 0.599). Qualitative analysis revealed patterns underlying performance differences between BOW and rule-based approaches. CONCLUSION: NLP holds promise for identifying EHR-documented goals-of-care discussions. However, the rarity of goals-of-care content in EHR data limits performance. Our findings highlight opportunities to optimize NLP modeling approaches, and support further exploration of different NLP approaches to identify goals-of-care discussions.

Feasibility of implementing a same-day electronic screening tool for clinical assessment to measure patient-reported outcomes for eliciting actionable information on adherence to HIV medication and related factors in a busy Canadian urban HIV clinic.

BACKGROUND: An optimal adherence to antiretroviral therapy (ART) is fundamental for suppression of HIV viral load and favourable treatment outcomes. Patient-reported outcomes (PROs) are effective tools for improving patient-provider communication and focusing providers' awareness on current health problems. The objectives of this analysis were (1) to determine the feasibility of implementing an electronic screening tool to measure PROs in a Canadian HIV clinic to obtain information on ART adherence and related factors and (2) to determine the factors related to sub-optimal adherence. METHODS: This implementation research with a convenience sample of 600 people living with HIV (PLWH) was conducted in a busy, academic, urban HIV clinic in Toronto, Canada. PLWH were approached to participate in PRO assessments just prior to their in-clinic appointments, including health-related domains such as mental health, housing, nutrition, financial stress and medication adherence, and responses were summarized on a single sheet available for providers to review. Feasibility of implementing PROs was assessed by quantifying response rate, completion rate, time taken and participation rate. Medication adherence was elicited by self-report of the percentage of prescribed HIV medications taken in the last month. Unadjusted and adjusted odds ratios were estimated from logistic regression models to identify factors associated with adherence of <95%. RESULTS: Of the 748 PLWH invited to participate, 692 (participation rate: 92.5%) completed the PRO assessments as standard of care in clinic. Of these, 600 consented to the use of their PRO results for research and were included in this analysis. The average response rate to the ART-related questions was 96.8% and mean completion rate was 95.5%. The median time taken to complete the assessment was 12.0 (IQR = 8.4-17.3) min, adjusted 8.7 (IQR = 7.2-10.8) min. 445 (74.9%) of participants were male, and 153 (26.2%) reported dissatisfaction with ART. 105 (19.7%) of the PLWH reported ART adherence of <95%. Multivariable logistic regression identified the following risk factors for sub-optimal adherence: dissatisfaction with ART (OR = 2.30, 95% CI 1.38-3.83), not having a family doctor or not visiting a family doctor in last year (OR = 1.69, 95% CI 1.02-2.79). CONCLUSION: Collecting self-reported health information from PLWH through PROs in a busy urban clinic was feasible and can provide relevant information to healthcare providers on issues related to adherence. This has a potential to help in individualizing ambulatory care.

A learning health systems approach to integrating electronic patient-reported outcomes across the health care organization.

INTRODUCTION: Foundational to a learning health system (LHS) is the presence of a data infrastructure that can support continuous learning and improve patient outcomes. To advance their capacity to drive patient-centered care, health systems are increasingly looking to expand the electronic capture of patient data, such as electronic patient-reported outcome (ePRO) measures. Yet ePROs bring unique considerations around workflow, measurement, and technology that health systems may not be poised to navigate. We report on our effort to develop generalizable learnings that can support the integration of ePROs into clinical practice within an LHS framework. METHODS: Guided by action research methodology, we engaged in iterative cycles of planning, acting, observing, and reflecting around ePRO use with two primary goals: (1) mobilize an ePRO community of practice to facilitate knowledge sharing, and (2) establish guidelines for ePRO use in the context of LHS practice. Multiple, emergent data collection activities generated generalizable guidelines that document the tangible best practices for ePRO use in clinical care. We organized guidelines around thematic areas that reflect LHS structures and stakeholders. RESULTS: Three core thematic areas (and 24 guidelines) emerged. The theme of governance reflects the importance of leadership, knowledge management, and facilitating organizational learning around best practice models for ePRO use. The theme of integration considers the intersection of workflow, technology, and human factors for ePROs across areas of care delivery. Lastly, the theme of reporting reflects critical considerations for curating data and information, designing system functions and interactions, and presentation of ePRO data to support the translation of knowledge to action. CONCLUSIONS: The guidelines produced from this work highlight the complex, multidisciplinary nature of implementing change within LHS contexts, and the value of action research approaches to enable rapid, iterative learning that leverages the knowledge and experience of communities of practice.